Medical image processing apparatus, method of driving medical image processing apparatus, medical imaging system, and medical signal acquisition system

ABSTRACT

A medical image processing apparatus includes an image processing unit configured to acquire a first fluorescence image captured between a timing when a first drug is administered and a first timing when a second drug is administered and a fluorescence intensity of a predetermined region starts to increase, and a second fluorescence image captured after the first timing and generate an output image in which fluorescence generated by the second drug is enhanced on a basis of the first fluorescence image and the second fluorescence image.

FIELD

The present disclosure relates to a medical image processing apparatus,a method of driving the medical image processing apparatus, a medicalimaging system, and a medical signal acquisition system.

BACKGROUND

With the development of surgical techniques, surgical instruments, andthe like, surgery (so-called microsurgery) in which various types oftreatments are performed while an affected part is observed with amedical observation device such as an endoscope, a surgical microscope,or the like has been frequently performed. In addition, among suchmedical observation devices, not only a device that can opticallyobserve an affected part but also a device and a system that display animage of the affected part captured by an imaging device (a camera) orthe like on a display device such as a monitor as an electronic imagehave been proposed.

Furthermore, in recent years, as an observation method using anobservation device such as an endoscope or a surgical microscope, notonly a method of observing an operative field by light in a visiblelight band but also various types of observation methods called speciallight observation have been proposed. Examples of the special lightobservation include narrow band imaging (NBI), fluorescence imaging(FI), infra-red imaging (IRI), and the like.

As a specific example, in the fluorescence imaging, a fluorescentmaterial (in other words, a phosphor) with an affinity for a lesion suchas a cancer is administered to a test subject (a patient) in advance,excitation light for exciting the fluorescent material is irradiated,and the lesion is observed using a fluorescence image (that is, anobservation image based on a detection result of fluorescence) offluorescence emitted from the fluorescent material accumulated in thelesion. For example, Patent Literature 1 discloses an outline offluorescence imaging using indocyanine green (ICG) as a fluorescentmaterial and an example of surgery based on the fluorescence imaging.

CITATION LIST Non Patent Literature

Non Patent Literature 1: Mitsuo Kusano, “All about ICG FluorescenceNavigation Surgery”, INTER MEDICA Co., Ltd., 2008, p.265

SUMMARY Technical Problem

Meanwhile, when a drug containing a fluorescent material (hereinafter,also referred to as “fluorescent agent”) is administered andfluorescence imaging is performed, the fluorescent material (forexample, ICG) adheres to a blood vessel wall, a protein in a tissue, orthe like through a blood flow, and in some cases, fluorescence isemitted for a while. Due to such properties, under a situation in whicha fluorescent agent is administered a plurality of times, at the time ofthe second and subsequent administrations of the fluorescent agent, afluorescent material as a result of the previous administration and afluorescent material as a result of the new administration are mixed,and visibility may decrease. In order to handle such a situation, as anexample of a method of ensuring visibility, there is a method ofperforming observation after the fluorescent material remaining in anobservation region is released (washed out). However, in a case wherethis method is applied, it takes time before the fluorescent material isreleased, and as a result, the time required for observation tends to belonger.

Therefore, the present disclosure proposes a technology that enables animage acquired in response to a drug to be observed in a more suitablemanner even in a situation in which the drug is administered a pluralityof times.

Solution to Problem

According to the preseent disclosure, a medical image processingapparatus comprises an image processing unit configured to acquire afirst fluorescence image captured between a timing when a first drug isadministered and a first timing when a second drug is administered and afluorescence intensity of a predetermined region starts to increase, anda second fluorescence image captured after the first timing and generatean output image in which fluorescence generated by the second drug isenhanced on a basis of the first fluorescence image and the secondfluorescence image.

Furthermore, acccording to the present disclosure, a method of driving amedical image processing apparatus comprises causing a computer toacquire a first fluorescence image captured between a timing when afirst drug is administered and a first timing when a second drug isadministered and a fluorescence intensity of a predetermined regionstarts to increase, and a second fluorescence image captured after thefirst timing, and generate an output image in which fluorescencegenerated by the second drug is enhanced on a basis of the firstfluorescence image and the second fluorescence image.

Moreover, acccording to the present disclosure, a medical imaging systemcomprises: a light source configured to emit excitation light of afluorescent material contained in a drug to be administered to apatient; an imaging unit configured to receive and capture an image oflight including fluorescence generated by the drug; and an imageprocessing apparatus configured to generate an output image in whichfluorescence generated by a second drug is enhanced on a basis of afirst fluorescence image captured between a timing when a first drug isadministered and a first timing when the second drug is administered anda fluorescence intensity of a predetermined region starts to increase,and a second fluorescence image captured after the first timing.

Moreover, acccording to the present disclosure, a medical imageprocesses apparatus comprising an image processing unit configured tooutput an output image that corresponds to an imaging result of anaffected part of a patient by an imaging unit and in which lightbelonging to a wavelength band corresponding to a drug to beadministered to the patient is set as an imaging target, wherein theimage processing unit includes, as an operation mode, a first mode foroutputting an output image corresponding to an imaging result of theaffected part at that time, and a second mode for outputting an outputimage in which fluorescence generated by a second drug is enhanced on abasis of a first fluorescence image captured between a timing when afirst drug is administered and a first timing when the second drug isadministered and a fluorescence intensity of a predetermined regionstarts to increase, and a second fluorescence image captured after thefirst timing.

Moreover, acccording to the present disclosure, a medical signalacquisition system comprises: a light source configured to emit light ina wavelength band corresponding to a drug to be administered to apatient; an acquisition unit configured to acquire an optical signalbelonging to a wavelength band corresponding to the drug; and an opticalsignal extraction unit configured to extract an optical signal generatedby a second drug on a basis of a first optical signal acquired between atiming when a first drug is administered and a first timing when thesecond drug is administered and a fluorescence intensity of apredetermined region starts to increase, and a second optical signalacquired after the first timing.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a functional block diagram illustrating an example of afunctional configuration of a medical observation system according toone embodiment of the present disclosure.

FIG. 2 is a functional block diagram illustrating an example of thefunctional configuration of the medical observation system according tothe embodiment.

FIG. 3 is an explanatory diagram for explaining an outline of an exampleof signal processing on an image signal in the medical observationsystem according to the embodiment.

FIG. 4 is an explanatory diagram for explaining the outline of anexample of the signal processing on the image signal in the medicalobservation system according to the embodiment.

FIG. 5 is a flowchart illustrating an example of a flow of a series ofprocessing of the medical observation system according to theembodiment.

FIG. 6 is an explanatory diagram for explaining an example of themedical observation system according to the embodiment.

FIG. 7 is a block diagram illustrating an example of a functionalconfiguration of a medical observation system according to a firstmodification.

FIG. 8 is a block diagram illustrating an example of a functionalconfiguration of a medical observation system according to a secondmodification.

FIG. 9 is a functional block diagram illustrating a configurationexample of a hardware configuration of an information processingapparatus constituting the medical observation system according to theembodiment.

FIG. 10 is an explanatory diagram for explaining an application exampleof the medical observation system according to the embodiment.

FIG. 11 is an explanatory diagram for explaining another applicationexample of the medical observation system according to the embodiment.

FIG. 12 is an explanatory diagram for explaining yet another applicationexample of the medical observation system according to the embodiment.

DESCRIPTION OF EMBODIMENTS

Hereinafter, preferred embodiments of the present disclosure will bedescribed in detail with reference to the accompanying drawings. Notethat, in the present specification and the drawings, components havingsubstantially the same functional configuration are denoted by the samereference numerals, and redundant description is omitted.

Note that the description will be given in the following order.

1. Introduction

2. Technical Feature

2.1. Functional Configuration

2.2. Signal Processing

2.3. Flow of Processing

2.4. Example

2.5. Modification

3. Example of Hardware Configuration

4. Application Example

4.1. First Application Example: Microscope Imaging System

4.2. Second Application Example: Endoscopic Surgical System

5. Conclusion

1. INTRODUCTION

First, as an example of an observation method using a drug, amongobservation methods referred to as so-called special light observation,in particular, an outline of fluorescence imaging for observing afluorescence image of a lesion part by using a fluorescent material willbe described, and then technical problems of a medical observationsystem according to the present disclosure will be described.

In the fluorescence imaging, a fluorescent material with an affinity fora lesion such as a cancer is administered to a test subject (a patient)in advance, excitation light for exciting the fluorescent material isirradiated, and the lesion is observed using a fluorescence image (thatis, an observation image based on a detection result of fluorescence) offluorescence emitted from the fluorescent material accumulated in thelesion part.

A typical example of the fluorescent material used for fluorescenceimaging is indocyanine green (ICG). ICG emits fluorescence (that is,light in a near-infrared band) with a wavelength of around 820 nm byusing light with a wavelength of around 808 nm as excitation light.

Furthermore, in recent years, various types of fluorescent materialsother than ICG have been proposed as fluorescent materials used forfluorescence imaging from the viewpoints of, for example, the propertyof more selectively accumulating in a lesion such as a cancer, thereduction of the influence (the side effect) on a test subject as aresult of administration, and the like. In addition, among suchfluorescent materials, a fluorescent material that emits fluorescence ina wavelength band different from that of ICG and a fluorescent materialthat emits light belonging to a visible-light wavelength band have alsobeen proposed.

In the fluorescence imaging, for example, by observing the presence orabsence of fluorescence emitted from the fluorescent material such asICG and the temporal change of the fluorescence, the operator canvisually observe a blood flow and a lymph flow. In addition, byanalyzing the temporal change (that is, the temporal change of thefluorescent material flowing in the body of a patient) of thefluorescence emitted from the fluorescent material, for example, it isalso possible to discriminate between a portion with a good flow and aportion with a poor flow in a part where a desired medium (for example,blood or lymph) flows, such as a blood vessel or a lymph vessel.

Meanwhile, as a problem in a case where the fluorescence imaging usingthe fluorescent material such as ICG is used for surgery, whenfluorescence contrast imaging is performed by administering afluorescent agent, the fluorescent material adheres to a blood vesselwall, a protein in a tissue, or the like through a blood flow, and insome cases, fluorescence is emitted for a while. Under such a situation,in a case where the administration of the fluorescent agent and thefluorescence imaging are repeatedly performed in a relatively shortperiod of time, at the time of the administration of the fluorescentagent, a fluorescent material as a result of the previous administrationand a fluorescent material as a result of the new administration may bemixed. In such a case, since the fluorescence emitted by the fluorescentmaterial as a result of the previous administration remains, in somecases, the visibility of the fluorescence emitted by the fluorescentmaterial as a result of the new administration decreases. In order tohandle such a situation, as an example of a method of ensuringvisibility, there is a method of performing observation after thefluorescent material remaining in an observation region is released(washed out). However, in a case where this method is applied, it takestime before the fluorescent material is released, and as a result, thetime required for observation tends to be longer.

In view of the above circumstances, the present disclosure proposes atechnology that enables an image acquired in response to a drug to beobserved in a more suitable manner even in a situation in which the drugis administered a plurality of times.

Specifically, a technology is proposed that enables a fluorescence imagecorresponding to the imaging result of fluorescence emitted by afluorescent material as a result of the administration of a newfluorescent agent to be observed in a more suitable manner before therelease of the fluorescent material even under a situation in which theadministration of the fluorescent agent and fluorescence imaging arerepeatedly performed.

2. TECHNICAL FEATURE

Next, the technical features of the medical observation system accordingto one embodiment of the present disclosure will be described. Notethat, hereinafter, the technical features of the medical observationstem according to the present embodiment will be described byparticularly focusing on a case where a fluorescent agent containing apredetermined fluorescent material is administered to a patient and anaffected part is observed on the basis of a fluorescence imagecorresponding to the imaging result of fluorescence emitted by thefluorescent material.

2.1. Functional Configuration

First, an example of a functional configuration of the medicalobservation system according to the present embodiment will bedescribed. The medical observation system according to the presentembodiment is configured to be operable by selectively switching betweena normal observation mode and a difference display mode. The normalobservation mode is a mode for outputting a fluorescence imagecorresponding to the imaging result of an affected part at that time byan imaging unit (for example, an endoscope device, a microscope device,or the like). In addition, the difference display mode is a mode inwhich under a situation in which the administration of a fluorescentagent and fluorescence imaging are repeatedly performed, the influenceof fluorescence emitted by a fluorescent material previouslyadministered is reduced, so that the fluorescence image corresponding tothe imaging result of fluorescence emitted by a fluorescent materialnewly administered can be observed in a more suitable manner.Consequently, in this section, an example of a functional configurationof the medical observation system according to the present embodimentwill be described separately for the operation in the normal observationmode and for the operation in the difference display mode. Note that thenormal observation mode corresponds to an example of “first mode”, andthe difference display mode corresponds to an example of “second mode”.

First, an example of the functional configuration of the medicalobservation system according to the present embodiment in a case wherethe medical observation system operates in the normal observation modewill be described with reference to FIG. 1. FIG. 1 is a functional blockdiagram illustrating an example of a functional configuration of amedical observation system 11 according to the present embodiment, andillustrates an example of the functional configuration in a case wherethe medical observation system 11 operates in the normal observationmode. Note that, hereinafter, in a case where the medical observationsystem 11 at the time of the operation in the normal observation mode isparticularly described, it is also referred to as “medical observationsystem 11 a” for convenience. On the other hand, in a case where themedical observation system 11 at the time of the operation in thedifference display mode is particularly described, it is also referredto as “medical observation system 11 b” for convenience.

As illustrated in FIG. 1, the medical observation system 11 a includesan imaging unit 191, a signal processing unit 110, and an output unit193. Note that, in the following description, in a case where the signalprocessing unit 110 at the time of the operation in the normalobservation mode is particularly described, it is also referred to as“signal processing unit 110 a” for convenience as illustrated in FIG. 1.On the other hand, in a case where the signal processing unit 110 at thetime of the operation in the difference display mode is particularlydescribed, it is also referred to as “signal processing unit 110 b” forconvenience.

It is schematically illustrated that the imaging unit 191 receives lightfrom an affected part (an observation target) to capture an image of theaffected part. As a specific example, the imaging unit 191 can beconfigured as a part (for example, a camera head) of an endoscope deviceor a surgical microscope device. The imaging unit 191 outputs an imagecorresponding to the imaging result of the affected part to the signalprocessing unit 110 a. Note that, under a situation in whichfluorescence imaging is performed, an image signal (in other words, afluorescence image) corresponding to the imaging result of fluorescenceemitted by a fluorescent material contained in a fluorescent agentadministered to the patient is output from the imaging unit 191 to thesignal processing unit 110 a.

The output unit 193 presents various types of information to a user. Inthe medical observation system 11 according to the present embodiment,the output unit 193 is configured as a display unit such as a so-calleddisplay, and presents information to the user by displaying displayinformation such as an image (a still image or a moving image). As aspecific example, the output unit 193 may display an output image (afluorescence image) output from the signal processing unit 110 a to bedescribed later.

The signal processing unit 110 is configured as a so-called cameracontrol unit (CCU), generates an output image by performing varioustypes of image processing on an image signal (for example, afluorescence image) output from the imaging unit 191, and causes theoutput unit 193 to output the output image. As illustrated in FIG. 1,the signal processing unit 110 a includes a development processing unit111.

The development processing unit 111 performs predetermined imageprocessing (so-called development processing) on an image signal outputfrom the imaging unit 191 every predetermined unit period (for example,every frame) to generate an output image. In the example illustrated inFIG. 1, the development processing unit 111 sequentially outputs theoutput image generated to the output unit 193. As a result, the outputimage is presented to the user (for example, a doctor) via the outputunit 193.

An example of the functional configuration of the medical observationsystem according to the present embodiment in a case where the medicalobservation system operates in the normal observation mode has beendescribed with reference to FIG. 1.

Next, an example of the functional configuration of the medicalobservation system according to the present embodiment in a case wherethe medical observation system operates in the difference display modewill be described with reference to FIG. 2. FIG. 2 is a functional blockdiagram illustrating an example of the functional configuration of themedical observation system according to the present embodiment, andillustrates an example of the functional configuration in a case wherethe medical observation system operates in the difference display mode.

As illustrated in FIG. 2, the medical observation system 11 b includesthe imaging unit 191, the signal processing unit 110 b, and the outputunit 193. Note that the imaging unit 191 and the output unit 193 aresimilar to the example illustrated in FIG. 1, and thus detaileddescription thereof is omitted. Furthermore, the signal processing unit110 corresponds to an example of “image processing unit”. Moreover, adevice including the signal processing unit 110 corresponds to anexample of “medical image processing apparatus”.

As illustrated in FIG. 2, the signal processing unit 110 b includes thedevelopment processing unit 111, a motion estimation unit 112, adifference extraction unit 113, an addition unit 114, a frame memory115, and an alignment processing unit 116. Note that the developmentprocessing unit 111 is substantially similar to the example illustratedin FIG. 1, and thus detailed description thereof is omitted.

The motion estimation unit 112 compares a plurality of images (forexample, images corresponding to imaging results in different frames)corresponding to imaging results at different timings, the images beingoutput from the development processing unit 111, to estimate a relativemotion (in other words, a motion of an affected part in the image)between the imaging unit 191 and a subject (for example, an affectedpart to be observed). As a specific example, the motion estimation unit112 calculates a motion vector (that is, the direction and the amount ofchange of a motion) between timings (for example, between frames)corresponding to the plurality of images every predetermined unit data(for example, on a pixel basis). Examples of a motion estimation methodinclude a block matching method and a gradient method. Furthermore, themotion estimation unit 112 may improve the accuracy of motion estimationby estimating the motion of the affected part in the image between theimages on the basis of each of a plurality of motion estimation methodsand combining the respective estimation results. The motion estimationunit 112 then outputs the estimation result of the motion of theaffected part in the image to the difference extraction unit 113 and thealignment processing unit 116.

On the basis of the estimation result of the motion of the affected partin the image output from the motion estimation unit 112, the differenceextraction unit 113 calculates (extracts) a difference between theplurality of images corresponding to the imaging results at differenttimings, the images being output from the development processing unit111, to generate a difference image. Note that, in the followingdescription, for convenience, it is assumed that the difference image isgenerated for an image (hereinafter, also referred to as “current frameimage”) corresponding to the imaging result in the current frame and animage (hereinafter, also referred to as “previous frame image”) capturedin a frame before the current frame.

As a specific example, the difference extraction unit 113 shifts (thatis, corrects) the previous frame image on a pixel basis on the basis ofthe estimation result of the motion of the affected part in the image toperform an alignment between the current frame image and the previousframe image. The difference extraction unit 113 then calculates(extracts) a difference (for example, an inter-frame difference) betweenthe current frame image and the previous frame image after thealignment, and generates a difference image corresponding to thecalculation result of the difference.

As described above, the difference extraction unit 113 generates adifference image on the basis of images (for example, fluorescenceimages) sequentially output from the development processing unit 111,and outputs the difference image to the addition unit 114.

The frame memory 115 temporarily holds an output image output from theaddition unit 114 to be described later. The output image held in theframe memory 115 is read at a desired timing by the alignment processingunit 116 to be described later.

In addition, on the basis of a predetermined trigger, the frame memory115 excludes the output image held until the trigger is received fromthe output target to the alignment processing unit 116. As a specificexample, by discarding the output image held until the trigger isreceived on the basis of a predetermined trigger, the output image maybe excluded from the output target to the alignment processing unit 116.In addition, the frame memory 115 may separately hold the output imageheld until the trigger is received, thereby excluding the output imagefrom the output target to the alignment processing unit 116. As aresult, both the past output image separately held and the output imagenewly held in the frame memory 115 can also be set as output targets.

Note that the timing when the trigger is generated and the type of thetrigger are not particularly limited. For example, by using a change inthe operation mode of the medical observation system 11 as a trigger,the output image held by the frame memory 115 may be excluded from theoutput target to the alignment processing unit 116. As a more specificexample, by using the transition to the difference display mode as atrigger, the frame memory 115 may exclude the output image held untilthe trigger is received from the output target to the alignmentprocessing unit 116. Note that the frame memory 115 corresponds to anexample of “storage unit”.

The alignment processing unit 116 reads the output image (that is, thepast output image) held in the frame memory 115, and shifts (that is,corrects) the output image on the basis of the estimation result of themotion of the affected part in the image output from the motionestimation unit 112. The alignment processing unit 116 then outputs theshifted output image to the addition unit 114. As a result, thealignment is performed between the difference image output from thedifference extraction unit 113 to the addition unit 114 and the pastoutput image output from the alignment processing unit 116 to theaddition unit 114.

The addition unit 114 adds the difference image output from thedifference extraction unit 113 to the past output image output from thealignment processing unit 116 to newly generate an output image (thatis, an output image corresponding to the current frame). The additionunit 114 then causes the output unit 193 to output the output imagegenerated. Furthermore, the addition unit 114 causes the frame memory115 to hold the output image. That is, as described above, the outputimage held in the frame memory 115 is added to the difference imagegenerated thereafter as the past output image, and thus a new outputimage is generated. As described above, in the example illustrated inFIG. 2, the difference image is accumulated in the frame memory 115, andthe output image corresponding to the accumulation result is output viathe output unit 193. Note that the difference image output from thedifference extraction unit 113 corresponds to an example of “firstdifference image”, and the difference image accumulated in the framememory 115 corresponds to an example of “second difference image”.

An example of the functional configuration of the medical observationsystem according to the present embodiment in a case where the medicalobservation system operates in the difference display mode has beendescribed above with reference to FIG. 2.

Note that the above is only an example, and the functional configurationof the medical observation system 11 is not necessarily limited to theexample illustrated in FIGS. 1 and 2 as long as each function describedabove can be implemented. As a specific example, two or moreconfigurations among the respective configurations of the medicalobservation system 11 may be integrally configured. Furthermore, asanother example, some of the configurations of the signal processingunit 110 may be provided outside the signal processing unit 110.Further, the configuration related to the generation of an image basedon an image signal in the signal processing unit 110, such as thedevelopment processing unit 111, may be provided outside the signalprocessing unit 110.

Furthermore, although not illustrated in FIGS. 1 and 2, the medicalobservation system 11 may include a light source that emits lightbelonging to a wavelength band corresponding to a drug to beadministered to a patient. As a specific example, the light source maybe configured to be capable of emitting excitation light of afluorescent material contained in a fluorescent agent to be administeredto the patient. With such a configuration, it is possible to excite thefluorescent material contained in the fluorescent agent with the lightemitted from the light source, thereby causing the fluorescent materialto emit fluorescence.

In addition, the medical observation system 11 may be configured to becapable of outputting both a fluorescence image that is output at thetime of the operation in the normal observation mode and a fluorescenceimage that is output at the time of the operation in the differenceobservation mode. In this case, for example, in the medical observationsystem 11 illustrated in FIG. 2, it suffices that the image output fromthe development processing unit 111 is separately output to the outputunit 193 as an output image.

2.2. Signal Processing

Next, an example of signal processing (that is, image processing) on animage signal in the medical observation system according to the presentembodiment will be described in more detail with reference to FIGS. 3and 4. FIGS. 3 and 4 are explanatory diagrams for explaining an outlineof an example of the signal processing on the image signal in themedical observation system according to the present embodiment.

For example, FIG. 3 schematically illustrates a time-series change inthe luminance (that is, a change in the fluorescence intensity) of apredetermined part (for example, a predetermined pixel) of an image in acase where a fluorescent agent is administered a plurality of times atdifferent timings under a situation in which fluorescence imaging isperformed. In the example illustrated in FIG. 3, the horizontal axisrepresents the time, and the vertical axis represents the luminancevalue (that is, the fluorescence intensity). That is, the graph of FIG.3 illustrates a time-series change in the fluorescence intensity of apredetermined area (hereinafter, also referred to as “pixel of interest”for convenience) in the image. Furthermore, in FIG. 3, a timing t1indicates a timing corresponding to the detection result of a firstadministration of the fluorescent agent, and ideally corresponds to thetiming when the fluorescent agent is administered for the first time.Further, a timing t2 indicates a timing corresponding to the detectionresult of a second administration of the fluorescent agent, and ideallycorresponds to the timing when the fluorescent agent is administered forthe second time. Moreover, a timing t3 indicates the output timing of afluorescence image corresponding to the second administration result ofthe fluorescent agent.

As illustrated in FIG. 3, when the fluorescent agent is administered forthe first time at the timing t1, the fluorescent material flows into theregion (for example, the site of interest) corresponding to the pixel ofinterest with the diffusion of the fluorescent material, so that afluorescence intensity I of the pixel of interest increases in timeseries. Furthermore, with the start of emission (washout) of thefluorescent material from the region (the site of interest)corresponding to the pixel of interest, the fluorescence intensity I ofthe pixel of interest decreases in time series. Next, when theadministration of the fluorescent agent is performed for the second timeat the timing t2, as the fluorescent material newly administereddiffuses, the fluorescence intensity I of the pixel of interestincreases again in time series. Note that, in FIG. 3, a referencenumeral I₁ indicates the fluorescence intensity of the pixel of interestat the timing t2. That is, the component exceeding the fluorescenceintensity I₁ after the timing t2 corresponds to a change in thefluorescence intensity caused by the detection result of thefluorescence emitted by the fluorescent material contained in thefluorescent agent administered for the second time.

Meanwhile, as in the example illustrated in FIG. 3, after the timing t2when the second administration of the fluorescent agent is performed,the detection result of the fluorescence emitted by the fluorescentmaterial contained in the fluorescent agent administered for the firsttime is apparent. That is, at the time of the operation in the normalobservation mode, after the timing t2, the fluorescence intensitycorresponding to the detection result of the fluorescence emitted by thenewly administered fluorescent material is further added with thefluorescence intensity I₁ as a reference point. That is, the entirescreen may become brighter due to the influence of the detection resultof the fluorescence emitted by the fluorescent material administered forthe first time, and as a result, the visibility may decrease.

On the other hand, as described above, at the time of the operation inthe difference display mode, for example, the output image held in theframe memory 115 is discarded on the basis of a predetermined trigger(that is, the frame memory 115 is reset). For example, in the case ofthe example illustrated in FIG. 3, it is assumed that the frame memory115 is reset at the timing t2 on the basis of the detection result ofthe second administration of the fluorescent agent. In this case, theoutput image is generated on the basis of the component exceeding thefluorescence intensity I₁ after the timing t2.

Specifically, in a case where the change in fluorescence intensitybetween frames is indicated by ΔI, the change in fluorescence intensityfrom the timing t2 to the timing t3 is expressed by a formula shown as(Formula 1) below.

$\begin{matrix}{I = {\sum\limits_{t2}^{t3}{\Delta I}}} & (1)\end{matrix}$

As described above, at the time of the operation in the differencedisplay mode, the image that is output after the second administrationof the fluorescent agent is generated on the basis of (Formula 1)described above.

That is, in this case, the output image is generated on the basis of thecomponent exceeding the fluorescence intensity I₁ after the timing t2.

Here, an example of an output image presented in each of the normalobservation mode and the difference display mode will be described withreference to FIG. 4. In FIG. 4, a reference numeral V101 schematicallyindicates an output image (a fluorescence image) immediately after thefirst administration of a fluorescent agent, that is, an output image atthe timing t1 illustrated in FIG. 3. In addition, a reference numeralV103 schematically indicates an output image (a fluorescence image)immediately before the second administration of the fluorescent agent,that is, an output image immediately before the timing t2 illustrated inFIG. 3.

In addition, a reference numeral V105 schematically indicates an outputimage (a fluorescence image) immediately after the second administrationof the fluorescent agent, that is, an output image immediately after thetiming t2 illustrated in FIG. 3, at the time of the operation in thenormal observation mode. Moreover, a reference numeral V107schematically indicates an output image (a fluorescence image) that isoutput after the second administration of the fluorescent agent, thatis, an output image that is output at the timing t3 illustrated in FIG.3, at the time of the operation in the normal observation mode.

At the time of the operation in the normal observation mode, when thesecond administration of the fluorescent agent is performed, thedetection result (the fluorescence intensity) of fluorescence emitted bya fluorescent material contained in the newly administered fluorescentagent is added on the output image V103 immediately before theadministration. That is, in the output images V105 and V107 that areoutput thereafter, since the detection result of the fluorescencecorresponding to the first administration of the fluorescent agentremains, the entire image is brighter than the output image V103, and isa whitish image as a whole as illustrated in FIG. 4, for example. As aresult, at the time of the operation in the normal observation mode, forexample, a decrease in contrast, blown out highlights, or the like mayoccur in the output image presented after the second administration ofthe fluorescent agent, and the visibility may decrease.

On the other hand, a reference numeral V109 schematically indicates anoutput image (a fluorescence image) immediately after the secondadministration of the fluorescent agent, that is, an output imageimmediately after the timing t2 illustrated in FIG. 3, at the time ofthe operation in the difference display mode. Moreover, a referencenumeral V111 schematically indicates an output image (a fluorescenceimage) that is output after the second administration of the fluorescentagent, that is, an output image that is output at the timing t3illustrated in FIG. 3, at the time of the operation in the differencedisplay mode.

In the example illustrated in FIGS. 3 and 4, at the time of theoperation in the difference display mode, the frame memory 115 is resetat the timing of the second administration of the fluorescent agent, asillustrated as the output image V109. For this reason, in this case, thedetection result of fluorescence in the part where the fluorescenceintensity has increased after the second administration of thefluorescent agent is reflected in the output image. As a result, asillustrated as the output image V111, it is possible to ensurevisibility equivalent to that at the time of observation after the firstadministration of the fluorescent agent, even at the time of observationafter the second administration of the fluorescent agent.

Note that, in the present disclosure, in a case where a drug isadministered a plurality of times, a drug administered first correspondsto an example of “first drug”, and a drug administered after the firstdrug corresponds to an example of “second drug”. Specifically, in theexample illustrated in FIGS. 3 and 4, the fluorescent agent administeredat the timing t1 corresponds to an example of “first drug”, and thefluorescent agent administered at the timing t2 corresponds to “seconddrug”. Note that, in the example illustrated in FIG. 3, it is assumedthat the first drug and the second drug are the same type of drug, butthe first drug and the second drug may be different types of drugs. Inaddition, the fluorescence image captured between the timing when thefirst drug is administered and the timing when the second drug isadministered corresponds to an example of “first fluorescence image”,and a fluorescence image captured after the timing when the second drugis administered corresponds to an example of “second fluorescenceimage”. Specifically, in the example illustrated in FIG. 4, thefluorescence image V103 corresponds to an example of “first fluorescenceimage”, and the fluorescence image V111 corresponds to an example of“second fluorescence image”. That is, according to the medicalobservation system of the present embodiment, it is possible togenerate, as the output image, the fluorescence image V111 in which thefluorescence generated by the second drug (for example, the fluorescentagent administered for the second and subsequent times) is enhanced.

Note that, in the example illustrated in FIG. 3, the method of detectingthe timing when the fluorescent agent is administered, which triggersthe reset of the frame memory 115, is not particularly limited. As aspecific example, in a case where a predetermined operation is performedby a user in accordance with the administration of the fluorescentagent, the signal processing unit 110 may detect the operation to detectthe timing when the fluorescent agent is administered.

Furthermore, as another example, the signal processing unit 110 mayperform image analysis on an image (a fluorescence image) correspondingto the imaging result of the imaging unit 191 to detect the timing whenthe fluorescent agent is administered. As a specific example, in thecase of the example illustrated in FIG. 3, it is possible to detect thetiming t1 of the first administration of the fluorescent agent bydetecting the rise of the luminance value (the fluorescence intensity).In addition, by detecting the minimum value of the change in luminancevalue, that is, the timing when the luminance value decreasing in timeseries starts increasing again, it is possible to detect the timing ofthe second and subsequent administrations of the fluorescent agent (forexample, the timing t2 of the second administration of the fluorescentagent). Note that the timing when the luminance value decreasing in timeseries starts increasing again corresponds to an example of “firsttiming”.

It is needless to mention that the above is only an example, and themethod is not particularly limited as long as the signal processing unit110 can detect the timing of the administration of the fluorescentagent. In addition, although an example of a case where fluorescenceimaging is performed has been described above, even in a case where anobservation method other than fluorescence imaging is applied, themethod is not particularly limited as long as it is possible to detectthe timing when the drug to be used is administered, and the detectionmethod may be selectively switched according to the observation methodor the drug to be used. Furthermore, the above is only an example, anddoes not necessarily limit the function of the medical observationsystem according to the present embodiment. For example, in acquiring animage corresponding to the first fluorescence image or the secondfluorescence image, it is not always necessary to explicitly detect thetiming when the first drug or the second drug is administered. As aspecific example, in the example illustrated in FIG. 3, by setting animage (a fluorescence image) corresponding to the imaging result of thetransition of the fluorescence intensity in which the fluorescenceintensity rises, falls, and rises again as the first fluorescence image,it is possible to generate the output image without explicitly detectingthe timing of the administration of the drug.

Furthermore, the above example has described an example of a case wherethe same type of fluorescent agent is administered a plurality of times,but the application range of the medical observation system according tothe present embodiment is not necessarily limited, and for example, aplurality of types of fluorescent agents may be administeredindividually for each observation. That is, as long as it is possible todetect each fluorescence emitted by the fluorescent material containedin each of the plurality of types of fluorescent agents, it is alsopossible to selectively use any fluorescent agent among the plurality oftypes of fluorescent agents for each of a plurality of observations.

Further, in the example illustrated in FIG. 4, the fluorescence imageV103 held in the frame memory 115 may be separately held without beingdiscarded at the timing of the second administration of the fluorescentagent. With such a configuration, it is also possible to output thefluorescence image V103 before the second administration of thefluorescent agent and the fluorescence image V111 after the secondadministration of the fluorescent agent in a comparable manner.

An example of the signal processing (that is, the image processing) onthe image signal in the medical observation system according to thepresent embodiment has been described in more detail with reference toFIGS. 3 and 4.

2.3. Flow of Processing

Next, an example of a flow of a series of processing of the medicalobservation system according to the present embodiment will be describedparticularly focusing on the processing of the signal processing unit.For example,

FIG. 5 is a flowchart illustrating an example of the flow of a series ofprocessing of the medical observation system according to the presentembodiment, and in particular, illustrates an example of an operation ina case where the administration of a fluorescent agent and fluorescenceimaging are repeatedly performed.

As illustrated in FIG. 5, when the first administration of thefluorescent agent is detected (S101), the signal processing unit 110generates an output image on the basis of a fluorescence imagecorresponding to the imaging result of the imaging unit 191, and causesthe output unit 193 to output the output image (S103). As a result, thefluorescence image corresponding to the detection result of fluorescenceemitted by a fluorescent material contained in the fluorescent agentadministered for the first time is presented via the output unit 193.

Next, when the nth (second and subsequent) administration of thefluorescent agent is detected (S105), the signal processing unit 110resets the frame memory 115 (S109) in a case where the operation mode isthe difference display mode (S107, YES). In this case, the signalprocessing unit 110 presents an output image corresponding to thedetection result of the fluorescence after the reset, that is, an outputimage corresponding to the detection result of the fluorescence emittedby the fluorescent material contained in the fluorescent agentadministered for the nth time (S111). As a specific example, the outputimage V111 in the example illustrated in FIG. 4 corresponds to anexample of the output image presented in this case.

On the other hand, when the operation mode is the normal mode (S107,NO), the signal processing unit 110 does not reset the frame memory 115(S109). In this case, the signal processing unit 110 presents an outputimage reflecting the detection result of fluorescence as a result of aseries of administrations of the fluorescent agent during the operationin the normal mode (S111). As a specific example, the output image V107in the example illustrated in FIG. 4 corresponds to an example of theoutput image presented in this case.

Note that although not clearly illustrated in the example illustrated inFIG. 5, the operation mode may be appropriately switched at a desiredtiming, and the method of switching the operation mode is notparticularly limited either. As a specific example, the signalprocessing unit 110 may switch the operation mode to the differencedisplay mode in a case where the operation mode is the normalobservation mode at the time of detection of the nth (the second andsubsequent) administration of the fluorescent agent (S015). In addition,even when the operation mode is switched, the signal processing unit 110may switch whether or not to reset the frame memory 115 (S109) accordingto the setting of the operation mode at the timing of performing theprocessing indicated by the reference numeral S107. As a result, theoutput image based on the setting of the operation mode at that time ispresented.

The signal processing unit 110 repeatedly performs a series ofprocessing indicated by the reference numerals S105 to S111 unless theend of observation is instructed (S113, NO). Then, when the end of theobservation is instructed (S113, YES), the signal processing unit 110ends the performance of the series of processing illustrated in FIG. 5.

An example of the flow of a series of processing of the medicalobservation system according to the present embodiment has beendescribed with reference to FIG. 5 particularly focusing on theprocessing of the signal processing unit.

2.4. Example

Next, an example of the medical observation system according to thepresent embodiment will be described. For example, FIG. 6 is anexplanatory diagram for explaining an example of the medical observationsystem according to the present embodiment, and illustrates an exampleof a case where blood flow evaluation is performed before and afteranastomosis of the intestine by fluorescence imaging using ICG as afluorescent material.

Specifically, as illustrated in the upper diagram of FIG. 6, the firstadministration of a fluorescent agent is performed first before theanastomosis of the intestine, and the blood flow before the anastomosisis evaluated on the basis of a fluorescence image V200 corresponding tothe detection result of fluorescence after the administration. Forexample, a reference numeral V201 schematically indicates a region inwhich the detection result (hereinafter, also referred to as “firstfluorescent component” for convenience) of fluorescence emitted from ICGcontained in the fluorescent agent administered for the first time isdetected. That is, the blood circulation state (for example, how far theblood flow has reached) is checked by the region V201 corresponding tothe first fluorescent component presented in the fluorescence imageV200, and the anastomosis site is determined according to the result ofthe check.

Subsequently, the second administration of the fluorescent agent isperformed after the anastomosis, and the blood flow after theanastomosis is evaluated on the basis of a fluorescence imagecorresponding to the detection result of fluorescence after theadministration.

For example, a fluorescence image V210 illustrated in the lower left ofFIG. 6 illustrates an example of an output image presented after thesecond administration of the fluorescent agent at the time of theoperation in the normal observation mode. Specifically, a referencenumeral V211 schematically indicates the anastomosis site. Further, areference numeral V213 schematically indicates a region corresponding tothe first fluorescent component. On the other hand, a reference numeralV215 schematically indicates a region in which the detection result(hereinafter, also referred to as “second fluorescent component” forconvenience) of fluorescence emitted from ICG contained in thefluorescent agent administered for the second time is detected. That is,in the fluorescence image V210 presented in the normal observation mode,the first fluorescent component remains in the fluorescence image V210,and the fluorescence intensity is added in the corresponding area V213.In view of such characteristics, it is difficult to determine which partin the fluorescence image V210 corresponds to the second fluorescentcomponent, and as a result, it may be difficult to compare the time ofthe first administration of the fluorescent agent (that is, beforeanastomosis) with the time of the second administration of thefluorescent agent (that is, after anastomosis).

On the other hand, a fluorescence image V220 illustrated in the lowerright of FIG. 6 illustrates an example of an output image presentedafter the second administration of the fluorescent agent at the time ofthe operation in the difference display mode. Specifically, a referencenumeral V221 schematically indicates the anastomosis site. Further, areference numeral V223 schematically indicates a region corresponding tothe second fluorescent component. That is, in the fluorescence imageV220 presented in the difference display mode, since the influence ofthe first fluorescent component is suppressed, the region V223corresponding to the second fluorescent component can be visuallyrecognized with the same visibility as the fluorescence image V200presented after the first administration of the fluorescent agent. Fromsuch characteristics, at the time of operation in the difference displaymode, it is possible to easily compare the time of the firstadministration of the fluorescent agent (that is, before anastomosis)with the time of the second administration of the fluorescent agent(that is, after anastomosis).

The example of the medical observation system according to the presentembodiment has been described above with reference to FIG. 6.

2.5. Modification

Next, as a modification of the medical observation system according tothe present embodiment, another example of the functional configurationof the medical observation system will be described.

(First Modification)

First, an example of a configuration of a medical observation systemaccording to a first modification will be described with reference toFIG. 7. FIG. 7 is a block diagram illustrating an example of afunctional configuration of the medical observation system according tothe first modification. Note that, in the following description, in acase where the medical observation system according to the presentmodification is particularly described, it is also referred to as“medical observation system 12” for convenience.

As illustrated in FIG. 7, the medical observation system 12 includes theimaging unit 191, a signal processing unit 120, and the output unit 193.Note that the output unit 193 is substantially similar to the outputunit 193 in the example illustrated in FIGS. 1 and 2, and thus detaileddescription thereof is omitted.

The imaging unit 191 is configured as a so-called dual plate camera,separates incident light into visible light and fluorescence (forexample, in the case of ICG, infrared light) emitted by a fluorescentmaterial, and detects the respective rays by different image sensors,thereby individually outputting detection results of the respectiverays. Note that the configuration for separating incident light intovisible light and fluorescence is not particularly limited. As aspecific example, incident light may be separated into visible light andfluorescence by using a color separation optical system configured usingan optical film that separates incident light according to wavelengthcharacteristics such as a dichroic film. On the basis of the aboveconfiguration, a fluorescence image and a visible light image areindividually output from the imaging unit 191 to the signal processingunit 120.

The signal processing unit 120 includes development processing units 121and 122, a motion estimation unit 123, a difference extraction unit 124,an addition unit 125, a frame memory 126, and an alignment processingunit 127. Note that the development processing unit 121, the differenceextraction unit 124, the addition unit 125, the frame memory 126, andthe alignment processing unit 127 are substantially similar to thedevelopment processing unit 111, the difference extraction unit 113, theaddition unit 114, the frame memory 115, and the alignment processingunit 116 in the example illustrated in FIG. 2, and thus detaileddescription thereof is omitted.

The development processing unit 122 performs predetermined imageprocessing (so-called development processing) on an image signalcorresponding to the detection result of visible light output from theimaging unit 191 every predetermined unit period (for example, everyframe) to generate a visible image. The development processing unit 122then outputs the visible light image generated on the basis of the imageprocessing to the motion estimation unit 123.

The motion estimation unit 123 estimates a relative motion (in otherwords, a motion of an affected part in the image) between the imagingunit 191 and a subject (for example, an affected part to be observed) onthe basis of the visible light image output from the developmentprocessing unit 122. Note that the method by which the motion estimationunit 123 estimates the motion of the affected part in the image issimilar to that of the motion estimation unit 112 in the exampleillustrated in FIG. 2 except that the type of image used for theestimation is different, and thus detailed description thereof isomitted. The motion estimation unit 123 then outputs the estimationresult of the motion of the affected part in the image to the differenceextraction unit 124 and the alignment processing unit 127.

As described above, in the medical observation system 12 according tothe present modification, the output image (for example, thefluorescence image) is generated according to the imaging result of theaffected part in which the light (for example, fluorescence) belongingto a first wavelength band corresponding to a drug (for example, afluorescent agent) to be used is set as an imaging target. On the otherhand, the motion of the affected part in the image is estimatedaccording to the imaging result of the affected part in which light (forexample, visible light) belonging to a second wavelength band differentfrom the first wavelength band is set as the imaging target. In thevisible light image, the subject (the affected part) tends to berecognized more clearly than in the fluorescence image. In view of suchcharacteristics, by using the visible light image for estimating themotion of the affected part in the image, the accuracy of the estimationcan be further improved as compared with the case of using thefluorescence image.

Note that the above is only an example, and does not necessarily limitthe configuration of the medical observation system 12 according to thepresent modification. As a specific example, the configuration of theimaging unit 191 is not limited to the example of the so-called dualplate camera as long as it is possible to individually capture images inwhich rays belonging to different wavelength bands are imaging targets,such as the fluorescence image and the visible light image. As aspecific example, the imaging unit 191 may be configured to capture thefluorescence image and the visible light image in a time divisionmanner.

An example of the configuration of the medical observation systemaccording to the first modification has been described with reference toFIG. 7.

(Second Modification)

Next, an example of a configuration of a medical observation systemaccording to a second modification will be described with reference toFIG. 8. FIG. 8 is a block diagram illustrating an example of afunctional configuration of the medical observation system according tothe second modification. Note that, in the following description, in acase where the medical observation system according to the presentmodification is particularly described, it is also referred to as“medical observation system 13” for convenience.

As illustrated in FIG. 7, the medical observation system 12 includes theimaging unit 191, a signal processing unit 130, and the output unit 193.Note that the imaging unit 191 and the output unit 193 are substantiallysimilar to the imaging unit 191 and the output unit 193 in the exampleillustrated in FIGS. 1 and 2, and thus detailed description thereof isomitted.

The signal processing unit 130 includes a development processing unit131, a frame memory 132, a motion estimation unit 133, an alignmentprocessing unit 134, and a difference extraction unit 135. Note that thedevelopment processing unit 131 is substantially similar to thedevelopment processing unit 111 in the example illustrated in FIGS. 1and 2, and thus detailed description thereof is omitted.

The development processing unit 131 outputs an image (a fluorescenceimage) corresponding to the development result every predetermined unitperiod (for example, every frame) to the motion estimation unit 133 andthe difference extraction unit 135. Note that, in the followingdescription, for convenience, it is assumed that the developmentprocessing unit 131 outputs a fluorescence image for each frame.

Furthermore, in the medical observation system 13 according to thepresent modification, the fluorescence image output from the developmentprocessing unit 131 at the timing when a predetermined trigger isreceived is held in the frame memory 132. Note that, in the followingdescription, the fluorescence image held in the frame memory 132 is alsoreferred to as “reference frame image”. Further, the fluorescence imagesequentially output from the development processing unit 131 to themotion estimation unit 133 and the difference extraction unit 135 isalso referred to as “current frame image”. That is, the reference frameimage corresponds to the fluorescence image output from the developmentprocessing unit 131 in the frame before the current frame image isoutput from the development processing unit 131. That is, in the presentmodification, the reference frame corresponds to an example of “firstfluorescence image”, and the current frame image corresponds to anexample of “second fluorescence image”.

The motion estimation unit 133 compares the current frame image outputfrom the development processing unit 131 with the reference frame imageheld in the frame memory 132 to estimate a relative motion (in otherwords, a motion of an affected part in an image) between the imagingunit 191 and a subject (for example, an affected part to be observed).That is, the motion of the affected part in the image estimated at thistime is based on the relationship in position and attitude between theimaging unit 191 and the subject, the relationship changing in theperiod between the timing when the reference frame image is captured andthe timing when the current frame image is captured. The motionestimation unit 133 then outputs the estimation result of the motion ofthe affected part in the image to the alignment processing unit 134.

The alignment processing unit 134 reads the reference frame image heldin the frame memory 132, and shifts the reference frame image on thebasis of the estimation result of the motion of the affected part in theimage output from the motion estimation unit 133. The alignmentprocessing unit 134 then outputs the shifted reference frame image tothe difference extraction unit 135. As a result, the alignment isperformed between the current frame image and the reference frame image.

The difference extraction unit 135 calculates (extracts) a differencebetween the current frame image output from the development processingunit 131 and the reference frame image after the alignment output fromthe alignment processing unit 134, and generates a difference imagecorresponding to the calculation result of the difference. Thedifference extraction unit 135 then causes the output unit 193 to outputthe difference image generated.

The difference image presented as described above is an image in whichthe influence of a fluorescent component captured as the reference frameimage, which becomes apparent in the current frame image, is suppressed.Using such characteristics, for example, under a situation in which theadministration of the fluorescent agent and the fluorescence imaging arerepeatedly performed, control may be executed such that the referenceframe image is held at the timing of the second and subsequentadministrations of the fluorescent agent. With such control, it is alsopossible to output a fluorescence image (for example, the fluorescenceimage V111 illustrated in FIG. 4) in which the influence of fluorescenceemitted by the fluorescent material contained in the previouslyadministered fluorescent agent is suppressed via the output unit 193.

Note that the above is only an example, and does not necessarily limitthe configuration of the medical observation system 12 according to thepresent modification. As a specific example, it is also possible tocombine the medical observation system 13 according to the presentmodification with the medical observation system 12 according to secondmodification described above. That is, in the medical observation system13 according to the present modification, the visible light image may beused for estimating the motion of the affected part in the image. Inthis case, it suffices that, with respect to the visible light image,the reference frame image is held in the frame memory 132 separatelyfrom the fluorescence image, and the motion of the affected part in theimage is estimated on the basis of the comparison between the currentframe image (the visible light image) and the reference frame image.

An example of the configuration of the medical observation systemaccording to the second modification has been described with referenceto FIG. 8.

3. EXAMPLE OF HARDWARE CONFIGURATION

Next, an example of a hardware configuration of an informationprocessing apparatus (for example, the signal processing unit 110illustrated in FIGS. 1 and 2 or the like) that performs various types ofprocessing in the medical observation system according to the presentembodiment will be described in detail with reference to FIG. 9. FIG. 9is a functional block diagram illustrating a configuration example of ahardware configuration of an information processing apparatusconstituting the medical observation system according to one embodimentof the present disclosure.

An information processing apparatus 900 constituting the medicalobservation system according to the present embodiment mainly includes aCPU 901, a ROM 902, and a RAM 903. In addition, the informationprocessing apparatus 900 further includes a host bus 907, a bridge 909,an external bus 911, an interface 913, an input device 915, an outputdevice 917, a storage device 919, a drive 921, a connection port 923,and a communication device 925.

The CPU 901 functions as an arithmetic processing unit and a controldevice, and controls all or part of the operation in the informationprocessing apparatus 900 according to various types of programs recordedin the ROM 902, the RAM 903, the storage device 919, or a removablerecording medium 927. The ROM 902 stores programs, arithmeticparameters, and the like used by the CPU 901. The RAM 903 primarilystores programs used by the CPU 901, parameters that appropriatelychange in the execution of the programs, and the like. These units aremutually connected by the host bus 907 including an internal bus such asa CPU bus. Note that each configuration of the signal processing unit110 illustrated in FIGS. 1 and 2, that is, the development processingunit 111, the motion estimation unit 112, the difference extraction unit113, the addition unit 114, and the alignment processing unit 116 can beimplemented by the CPU 901.

The host bus 907 is connected to the external bus 911 such as aperipheral component interconnect/interface (PCI) bus via the bridge909. In addition, the input device 915, the output device 917, thestorage device 919, the drive 921, the connection port 923, and thecommunication device 925 are connected to the external bus 911 via theinterface 913.

The input device 915 is an operation unit operated by a user, such as amouse, a keyboard, a touch panel, a button, a switch, a lever, and apedal. Furthermore, the input device 915 may be, for example, a remotecontrol unit (so-called remote controller) using infrared rays or otherradio waves, or an external connection device 929 such as a mobile phoneor a PDA compatible with the operation of the information processingapparatus 900. Further, the input device 915 includes, for example, aninput control circuit that generates an input signal on the basis ofinformation input by the user using the operation unit described aboveand outputs the input signal to the CPU 901. By operating the inputdevice 915, the user of the information processing apparatus 900 caninput various types of data to the information processing apparatus 900and instruct the information processing apparatus 900 on processingoperations.

The output device 917 includes a device capable of visually or aurallynotifying the user of information acquired. Examples of such a deviceinclude a display device such as a CRT display device, a liquid crystaldisplay device, a plasma display device, an EL display device, and alamp, an audio output device such as a speaker and a headphone, and aprinter device. The output device 917 outputs, for example, resultsobtained by various types of processing performed by the informationprocessing apparatus 900. Specifically, the display device displaysresults obtained by various types of processing performed by theinformation processing apparatus 900 as text or images. On the otherhand, the audio output device converts an audio signal includingreproduced audio data, acoustic data, or the like into an analog signaland outputs the analog signal. Note that the output unit 193 illustratedin FIGS. 1 and 2 can be implemented by the output device 917.

The storage device 919 is a data storage device configured as an exampleof a storage unit of the information processing apparatus 900. Thestorage device 919 includes, for example, a magnetic storage unit devicesuch as a hard disk drive (HDD), a semiconductor storage device, anoptical storage device, a magneto-optical storage device, or the like.The storage device 919 stores programs executed by the CPU 901, varioustypes of data, and the like. Note that the frame memory 115 illustratedin FIGS. 1 and 2 can be implemented by any one of the storage device 919and the RAM 903, or a combination thereof.

The drive 921 is a reader and writer for a recording medium, and isbuilt in or externally attached to the information processing apparatus900. The drive 921 reads information recorded in the removable recordingmedium 927 attached to the drive 921, such as a magnetic disk, anoptical disk, a magneto-optical disk, or a semiconductor memory, andoutputs the information to the RAM 903. Moreover, the drive 921 can alsowrite a record in the removable recording medium 927 attached to thedrive 921, such as a magnetic disk, an optical disk, a magneto-opticaldisk, or a semiconductor memory. The removable recording medium 927 is,for example, a DVD medium, an HD-DVD medium, a Blu-ray (registeredtrademark) medium, or the like. Furthermore, the removable recordingmedium 927 may be a CompactFlash (CF) (registered trademark), a flashmemory, a secure digital (SD) memory card, or the like. Further, theremovable recording medium 927 may be, for example, an integratedcircuit (IC) card on which a non-contact IC chip is mounted, anelectronic device, or the like.

The connection port 923 is a port for directly connecting to theinformation processing apparatus 900. Examples of the connection port923 include a universal serial bus (USB) port, an IEEE 1394 port, asmall computer system interface (SCSI) port, and the like. Otherexamples of the connection port 923 include an RS-232C port, an opticalaudio terminal, a high-definition multimedia interface (HDMI)(registered trademark) port, and the like. By connecting the externalconnection device 929 to the connection port 923, the informationprocessing apparatus 900 directly acquires various types of data fromthe external connection device 929 or provides various types of data tothe external connection device 929.

The communication device 925 is, for example, a communication interfaceincluding a communication device for connecting to a communicationnetwork (network) 931 or the like. The communication device 925 is, forexample, a communication card for wired or wireless local area network(LAN), Bluetooth (registered trademark), or wireless USB (WUSB), or thelike. Furthermore, the communication device 925 may be a router foroptical communication, a router for asymmetric digital subscriber line(ADSL), a modem for various types of communications, or the like. Forexample, the communication device 925 can transmit and receive signalsand the like to and from the Internet and other communication devicesaccording to a predetermined protocol such as TCP/IP. Moreover, thecommunication network 931 connected to the communication device 925includes a network connected in a wired or wireless manner or the like,and may be, for example, the Internet, a home LAN, infraredcommunication, radio wave communication, satellite communication, or thelike.

An example of the hardware configuration that can implement thefunctions of the information processing apparatus 900 constituting themedical observation system according to the embodiment of the presentdisclosure has been described above. Each of the components describedabove may be configured using a general-purpose member, or may beconfigured by hardware specialized for the function of each component.Consequently, it is possible to appropriately change the hardwareconfiguration to be used according to the technical level at the time ofcarrying out the present embodiment. Note that, although not illustratedin FIG. 9, it is needless to mention that various types ofconfigurations corresponding to the information processing apparatus 900constituting the medical observation system are included.

Note that a computer program for implementing each function of theinformation processing apparatus 900 constituting the medicalobservation system according to the present embodiment described abovecan be produced and mounted on a personal computer or the like.Furthermore, a computer-readable recording medium storing such acomputer program can also be provided. The recording medium is, forexample, a magnetic disk, an optical disk, a magneto-optical disk, aflash memory, or the like. Furthermore, the computer program describedabove may be distributed via, for example, a network without using arecording medium. Moreover, the number of computers that execute thecomputer program is not particularly limited. For example, a pluralityof computers (for example, a plurality of servers and the like) mayexecute the computer program in cooperation with each other.

4. APPLICATION EXAMPLE

Next, an application example of the medical observation system accordingto one embodiment of the present disclosure will be described.

4.1. First Application Example Microscope Imaging System

First, as a first application example, an example in a case where themedical observation system according to one embodiment of the presentdisclosure is configured as a microscope imaging system including amicroscope unit will be described with reference to FIG. 10.

FIG. 10 is an explanatory diagram for explaining an application exampleof the medical observation system according to one embodiment of thepresent disclosure, and illustrates an example of a schematicconfiguration of a microscope imaging system. Specifically, FIG. 10illustrates an example of a case where a surgical video microscopedevice including an arm is used as an application example of a casewhere the microscope imaging system according to one embodiment of thepresent disclosure is used.

For example, FIG. 10 schematically illustrates a state of an operationusing the surgical video microscope device. Specifically, referring toFIG. 10, a state where a doctor who is a practitioner (a user) 520 isperforming surgery on an operation target (a patient) 540 on anoperation table 830 using a surgical instrument 521 such as a scalpel,tweezers, or forceps is illustrated. Note that, in the followingdescription, the operation is a generic term for various types ofmedical treatment performed by the doctor who is the user 520 on thepatient who is the operation target 540, such as surgery andexamination. Furthermore, in the example illustrated in FIG. 10, a stateof surgery is illustrated as an example of the operation, but theoperation using a surgical video microscope device 510 is not limited tosurgery, and may be other various types of operations.

The surgical video microscope device 510 is provided beside theoperation table 830. The surgical video microscope device 510 includes abase part 511 that is is a base, an arm part 512 extending from the basepart 511, and an imaging unit 515 connected to a distal end of the armpart 512 as a distal end unit. The arm part 512 includes a plurality ofjoint parts 513 a, 513 b, and 513 c, a plurality of links 514 a and 514b connected by the joint parts 513 a and 513 b, and the imaging unit 515provided at the distal end of the arm part 512. In the exampleillustrated in FIG. 10, the arm part 512 includes three joint parts 513a to 513 c and two links 514 a and 514 b for the sake of simplicity, butin practice, the number and shape of the joint parts 513 a to 513 c andthe links 514 a and 514 b, the directions of the drive shafts of thejoint parts 513 a to 513 c, and the like may be appropriately set so asto achieve a desired degree of freedom in consideration of the degree offreedom of the position and attitude of the arm part 512 and the imagingunit 515.

The joint parts 513 a to 513 c have a function of rotatably connectingthe links 514 a and 514 b to each other, and the drive of the arm part512 is controlled by driving the rotation of the joint parts 513 a to513 c. Here, in the following description, the position of eachcomponent of the surgical video microscope device 510 means a position(coordinates) in a space defined for drive control, and the attitude ofeach component means a direction (an angle) with respect to any axis inthe space defined for drive control. Furthermore, in the followingdescription, the drive (or the drive control) of the arm part 512 meansthat the position and attitude of each component member of the arm part512 are changed (change is controlled) by driving (or drive control of)the joint parts 513 a to 513 c and driving (or drive control of) thejoint parts 513 a to 513 c.

The imaging unit 515 is connected to the distal end of the arm part 512as a distal end unit. The imaging unit 515 is a unit that acquires animage of an imaging target, and is, for example, a camera or the likecapable of capturing a moving image or a still image. As illustrated inFIG. 10, attitudes and positions of the arm part 512 and the imagingunit 515 are controlled by the surgical video microscope device 510 suchthat the imaging unit 515 provided at the distal end of the arm part 512captures a state of an operation site of the operation target 540. Notethat the configuration of the imaging unit 515 connected to the distalend of the arm part 512 as a distal end unit is not particularlylimited, and for example, the imaging unit 515 is configured as amicroscope that acquires an enlarged image of the imaging target.Furthermore, the imaging unit 515 may be configured to be removable fromthe arm part 512. With such a configuration, for example, the imagingunit 515 based on the use application may be appropriately connected tothe distal end of the arm part 512 as a distal end unit. Note that, asthe imaging unit 515, for example, an imaging device to which thebranching optical system according to the embodiment described above isapplied can be applied. That is, in the present application example, theimaging unit 515 or the surgical video microscope device 510 includingthe imaging unit 515 can correspond to an example of “medicalobservation device”. Further, the present description has been givenfocusing on the case where the imaging unit 515 is applied as a distalend unit, but the distal end unit connected to the distal end of the armpart 512 is not necessarily limited to the imaging unit 515.

Further, a display device 550 such as a monitor or a display isinstalled at a position facing the user 520. The image of the operationsite captured by the imaging unit 515 is displayed as an electronicimage on the display screen of the display device 550. The user 520performs various treatments while viewing the electronic image of theoperation site displayed on the display screen of the display device550.

With the above configuration, it is possible to perform surgery whileimaging the operation site by the surgical video microscope device 510.

Note that the present disclosure is not limited to the above, and thetechnology described above according to the present disclosure can beapplied within a range not departing from the basic idea of the medicalobservation system according to one embodiment of the presentdisclosure. As a specific example, the technology described aboveaccording to the present disclosure can be appropriately applied notonly to a system to which the endoscope or the operation microscopedescribed above is applied, but also to a system in which an image of anaffected part is captured by an imaging device of a desired form toenable observation of the affected part.

An example in a case where the medical observation system according toone embodiment of the present disclosure is configured as a microscopeimaging system including a microscope unit has been described as thefirst application example with reference to FIG. 10.

4.2. Second Application Example Endoscopic Surgical System

Next, as a second application example, an example in a case where themedical observation system according to one embodiment of the presentdisclosure is configured as an endoscopic surgical system including anendoscope unit will be described with reference to FIGS. 11 and 12.

FIGS. 11 and 12 are explanatory diagrams for explaining anotherapplication example of the medical observation system according to oneembodiment of the present disclosure, and illustrate an example of aschematic configuration of an endoscopic surgical system.

For example, FIG. 11 is a diagram illustrating an example of a schematicconfiguration of an endoscopic surgical system to which the technologyaccording to the present disclosure can be applied. FIG. 11 illustratesa state where an operator (a doctor) 667 is performing surgery on apatient 671 on a patient bed 669 using an endoscopic surgical system600. As illustrated, the endoscopic surgical system 600 includes anendoscope 601, other surgical tools 617, a support arm device 627 thatsupports the endoscope 601, and a cart 637 on which various devices forendoscopic surgery are mounted.

In endoscopic surgery, instead of cutting and opening the abdominalwall, a plurality of cylindrical puncture instruments called trocars 625a to 625 d are punctured into the abdominal wall. Then, a lens barrel603 of the endoscope 601 and the other surgical tools 617 are insertedinto the body cavity of the patient 671 from the trocars 625 a to 625 d.In the example illustrated, as the other surgical tools 617, apneumoperitoneum tube 619, an energy treatment tool 621, and forceps 623are inserted into the body cavity of the patient 671. Furthermore, theenergy treatment tool 621 is a treatment tool that performs incision anddetachment of tissue, sealing of a blood vessel, or the like byhigh-frequency current or ultrasonic vibration. However, the surgicaltools 617 illustrated are only an example, and various types of surgicaltools generally used in endoscopic surgery, such as tweezers and aretractor, may be used as the surgical tools 617.

An image of an operation site in the body cavity of the patient 671captured by the endoscope 601 is displayed on a display device 641.While viewing the image of the operation site displayed on the displaydevice 641 in real time, the operator 667 performs treatment such asresection of an affected part using the energy treatment tool 621 andthe forceps 623. Note that, although not illustrated, thepneumoperitoneum tube 619, the energy treatment tool 621, and theforceps 623 are supported by the operator 667, an assistant, or the likeduring surgery.

(Support Arm Device)

The support arm device 627 includes an arm part 631 extending from abase part 629. In the example illustrated, the arm part 631 includesjoint parts 633 a, 633 b, and 633 c and links 635 a and 635 b, and isdriven under the control of an arm control device 645. The endoscope 601is supported by the arm part 631, and its position and attitude arecontrolled. As a result, stable fixation of the position of theendoscope 601 can be achieved.

(Endoscope)

The endoscope 601 includes the lens barrel 603 whose region with apredetermined length from the distal end is inserted into the bodycavity of the patient 671, and a camera head 605 connected to theproximal end of the lens barrel 603. The example illustrates theendoscope 601 configured as a so-called rigid scope including the lensbarrel 603 that is rigid, but the endoscope 601 may be configured as aso-called flexible scope including the lens barrel 603 that is flexible.Note that the camera head 605 or the endoscope 601 including the camerahead 605 corresponds to an example of “medical observation device”.

An opening into which an objective lens is fitted is provided at thedistal end of the lens barrel 603. A light source device 643 isconnected to the endoscope 601, and light generated by the light sourcedevice 643 is guided to the distal end of the lens barrel by a lightguide extending inside the lens barrel 603, and is emitted toward anobservation target (in other words, an imaging target) in the bodycavity of the patient 671 via the objective lens. Note that theendoscope 601 may be a forward-viewing endoscope, an oblique-viewingendoscope, or a side-viewing endoscope.

An optical system and an imaging element are provided inside the camerahead 605, and reflected light (observation light) from the observationtarget is condensed on the imaging element by the optical system. Theobservation light is photoelectrically converted by the imaging element,and an electric signal corresponding to the observation light, that is,an image signal corresponding to an observation image is generated. Theimage signal is transmitted to a camera control unit (CCU) 639 as RAWdata. Note that the camera head 605 has a function of adjusting amagnification and a focal length by appropriately driving the opticalsystem.

Note that, for example, in order to support stereoscopic viewing (3Ddisplay) or the like, a plurality of imaging elements may be provided inthe camera head 605. In this case, a plurality of relay optical systemsare provided inside the lens barrel 603 in order to guide theobservation light to each of the plurality of imaging elements.

(Various Types of Devices Mounted on Cart)

The CCU 639 includes a central processing unit (CPU), a graphicsprocessing unit (GPU), and the like, and integrally controls theoperation of the endoscope 601 and the display device 641. Specifically,the CCU 639 performs, on the image signal received from the camera head605, various types of image processing for displaying an image based onthe image signal, such as development processing (demosaic processing).The CCU 639 provides the image signal subjected to the image processingto the display device 641. Furthermore, the CCU 639 transmits a controlsignal to the camera head 605 and controls the drive of the camera head605. The control signal can include information related to imagingconditions such as a magnification and a focal length.

The display device 641 displays an image based on the image signalsubjected to the image processing by the CCU 639 under the control ofthe CCU 639. In a case where the endoscope 601 supports high-resolutionimaging such as 4K (the number of horizontal pixels 3840×the number ofvertical pixels 2160) or 8K (the number of horizontal pixels 7680×thenumber of vertical pixels 4320), and/or in a case where the endoscope601 supports 3D display, a display device capable of high-resolutiondisplay and/or a display device capable of 3D display can be used as thedisplay device 641 corresponding to each case. In a case where theendoscope 601 supports high-resolution imaging such as 4K or 8K, afurther immersive feeling can be obtained by using a display device witha size of 55 inches or more as the display device 641. Furthermore, aplurality of the display devices 641 with different resolutions andsizes may be provided depending on the application.

The light source device 643 includes, for example, a light source suchas a light emitting diode (LED), and supplies irradiation light forcapturing an operation site to the endoscope 601.

The arm control device 645 includes, for example, a processor such as aCPU, and operates according to a predetermined program to control thedrive of the arm part 631 of the support arm device 627 according to apredetermined control system.

An input device 647 is an input interface for the endoscopic surgicalsystem 600. The user can input various types of information andinstructions to the endoscopic surgical system 600 via the input device647. For example, the user inputs various types of information relatedto surgery, such as physical information of a patient and informationrelated to the surgical method of the surgery, via the input device 647.Furthermore, for example, the user inputs an instruction to drive thearm part 631, an instruction to change imaging conditions (the type,magnification, focal length, and the like of irradiation light) of theendoscope 601, an instruction to drive the energy treatment tool 621,and the like via the input device 647.

The type of the input device 647 is not limited, and various known inputdevices may serve as the input device 647. As the input device 647, forexample, a mouse, a keyboard, a touch panel, a switch, a foot switch 657and/or a lever or the like can be applied. In a case where the touchpanel is used as the input device 647, the touch panel may be providedon the display surface of the display device 641.

Alternatively, the input device 647 may be, for example, a sensorincluded in a device worn by the user, such as a glasses-type wearabledevice or a head mounted display (HMD). In this case, various types ofinputs are performed according to the user's motion or line of sightdetected by these sensors. Further, the input device 647 includes acamera capable of detecting a motion of the user, and various types ofinputs are performed according to a gesture or a line of sight of theuser detected from a video captured by the camera. Furthermore, theinput device 647 includes a microphone capable of collecting a user'svoice, and various types of inputs are performed by voice via themicrophone. As described above, the input device 647 is configured to beable to input various types of information in a non-contact manner, andthus, in particular, the user (for example, the operator 667) belongingto a clean area can operate a device belonging to an unclean area in anon-contact manner. In addition, since the user can operate the devicewithout releasing his/her hand from the holding surgical tool, theconvenience of the user is improved.

A treatment-tool control device 649 controls the drive of the energytreatment tool 621 for cauterization and incision of tissues, sealing ofa blood vessel, or the like. A pneumoperitoneum device 651 feeds gasinto the body cavity of the patient 671 via the pneumoperitoneum tube619 in order to inflate the body cavity of the patient 671 for thepurpose of securing a field of view of the endoscope 601 and anoperating space of the operator. A recorder 653 is a device capable ofrecording various types of information related to surgery. A printer 655is a device capable of printing various types of information related tosurgery in various formats such as text, image, or graph.

Hereinafter, a particularly characteristic configuration of theendoscopic surgical system 600 will be described in more detail.

(Support Arm Device)

The support arm device 627 includes the base part 629 that is a base andthe arm part 631 extending from the base part 629. In the illustratedexample, the arm part 631 includes a plurality of the joint parts 633 a,633 b, and 633 c and a plurality of the links 635 a and 635 b connectedby the joint part 633 b, but in FIG. 11, the configuration of the armpart 631 is illustrated in a simplified manner for the sake ofsimplicity. In practice, the shape, number, and arrangement of the jointparts 633 a to 633 c and the links 635 a and 635 b, the directions ofthe rotation axes of the joint parts 633 a to 633 c, and the like can beappropriately set so that the arm part 631 has a desired degree offreedom. For example, the arm part 631 can be suitably configured tohave six degrees of freedom or more. As a result, since the endoscope601 can be freely moved within the movable range of the arm part 631,the lens barrel 603 of the endoscope 601 can be inserted into the bodycavity of the patient 671 from a desired direction.

An actuator is provided in each of the joint parts 633 a to 633 c, andthe joint parts 633 a to 633 c are configured to be rotatable around apredetermined rotation axis by driving the actuators. The drive of theactuator is controlled by the arm control device 645, so that therotation angle of each of the joint parts 633 a to 633 c is controlledand the drive of the arm part 631 is controlled accordingly. As aresult, control of the position and attitude of the endoscope 601 can beachieved. At this time, the arm control device 645 can control the driveof the arm part 631 by various types of known control methods such asforce control or position control.

For example, as the operator 667 appropriately performs an operationinput via the input device 647 (including the foot switch 657), thedrive of the arm part 631 may be appropriately controlled by the armcontrol device 645 in response to the operation input, and the positionand attitude of the endoscope 601 may be controlled. With this control,the endoscope 601 at the distal end of the arm part 631 can be movedfrom an arbitrary position to an arbitrary position and then fixedlysupported at the position after the movement. Note that the arm part 631may be operated by a so-called master-slave method. In this case, thearm part 631 can be remotely operated by the user via the input device647 installed at a place away from an operating room.

Furthermore, in a case where the force control is applied, the armcontrol device 645 may execute so-called power assist control ofreceiving an external force from the user and driving the actuator ofeach of the joint parts 633 a to 633 c so that the arm part 631 smoothlymoves in response to the external force. As a result, when the usermoves the arm part 631 while directly touching the arm part 631, the armpart 631 can be moved with a relatively light force. Consequently, it ispossible to more intuitively move the endoscope 601 with a simpleroperation, and the convenience of the user can be improved.

Here, in endoscopic surgery, the endoscope 601 is generally supported bya doctor called a scopist. On the other hand, by using the support armdevice 627, it is possible to more reliably fix the position of theendoscope 601 without manual operation, so that it is possible to stablyobtain an image of the operation site and smoothly perform the surgery.

Note that the arm control device 645 is not necessarily provided in thecart 637. Furthermore, the arm control device 645 is not necessarily asingle device. For example, the arm control device 645 may be providedin each of the joint parts 633 a to 633 c of the arm part 631 in thesupport arm device 627, and the drive control of the arm part 631 may beachieved by a plurality of the arm control devices 645 cooperating witheach other.

(Light Source Device)

The light source device 643 supplies the endoscope 601 with irradiationlight for capturing an operation site. The light source device 643includes, for example, a white light source including an LED, a laserlight source, or a combination thereof. At this time, in a case wherethe white light source is configured by a combination of RGB laser lightsources, the output intensity and output timing of each color (eachwavelength) can be controlled with high accuracy, so that the whitebalance of the captured image can be adjusted in the light source device643. Furthermore, in this case, by irradiating an observation targetwith laser light from each of the RGB laser light sources in a timedivision manner and controlling the drive of the imaging element of thecamera head 605 in synchronization with the irradiation timing, it isalso possible to capture an image corresponding to each RGB in a timedivision manner. According to this method, a color image can be obtainedwithout providing a color filter in the imaging element.

Further, the drive of the light source device 643 may be controlled soas to change the intensity of light to be output every predeterminedtime. By controlling the drive of the imaging element of the camera head605 in synchronization with the timing of the change of the lightintensity to acquire images in a time division manner and combining theimages, it is possible to generate an image of a high dynamic rangewithout so-called blocked up shadows and blown out highlights.

Furthermore, the light source device 643 may be configured to be able tosupply light in a predetermined wavelength band suitable for speciallight observation. In the special light observation, for example,so-called narrow band imaging is performed in which a predeterminedtissue such as a blood vessel in a mucosal surface layer is capturedwith high contrast by irradiating light in a narrower band thanirradiation light (that is, white light) at the time of normalobservation using wavelength dependency of light absorption in a bodytissue. Alternatively, in the special light observation, fluorescenceimaging that obtains an image by fluorescence generated by irradiationwith excitation light may be performed. In the fluorescence imaging, forexample, fluorescence from a body tissue can be observed by irradiatingthe body tissue with excitation light (autofluorescence imaging), or afluorescent image can be obtained by locally injecting a reagent such asindocyanine green (ICG) into a body tissue and irradiating the bodytissue with excitation light according to a fluorescence wavelength ofthe reagent. The light source device 643 can be configured to be able tosupply narrow band light and/or excitation light suitable for suchspecial light observation.

(Camera Head and CCU)

The functions of the camera head 605 and the CCU 639 of the endoscope601 will be described in more detail with reference to FIG. 12. FIG. 12is a block diagram illustrating an example of functional configurationsof the camera head 605 and the CCU 639 illustrated in FIG. 11.

Referring to FIG. 12, the camera head 605 includes a lens unit 607, animaging unit 609, a drive unit 611, a communication unit 613, and acamera head control unit 615 as functions thereof. Further, the CCU 639includes a communication unit 659, an image processing unit 661, and acontrol unit 663 as functions thereof. The camera head 605 and the CCU639 are connected to be bidirectionally communicable by a transmissioncable 665.

First, the functional configuration of the camera head 605 will bedescribed. The lens unit 607 is an optical system provided at aconnection portion with the lens barrel 603. Observation light takenfrom the distal end of the lens barrel 603 is guided to the camera head605 and enters the lens unit 607. The lens unit 607 is configured bycombining a plurality of lenses including a zoom lens and a focus lens.The optical characteristics of the lens unit 607 are adjusted so as tocondense the observation light on the light receiving surface of theimaging element in the imaging unit 609. Furthermore, the zoom lens andthe focus lens are configured to be movable in position on the opticalaxis in order to adjust the magnification and focal point of a capturedimage.

The imaging unit 609 includes an imaging element and is disposed at asubsequent stage of the lens unit 607. The observation light havingpassed through the lens unit 607 is condensed on the light receivingsurface of the imaging element, and an image signal corresponding to anobservation image is generated by photoelectric conversion. The imagesignal generated by the imaging unit 609 is provided to thecommunication unit 613.

As the imaging element constituting the imaging unit 609, for example, acomplementary metal oxide semiconductor (CMOS) image sensor or a chargecoupled device (CCD) image sensor that is capable of color imaging andhas a Bayer array, is used, but the imaging element may be an imagesensor for single-plate monochrome imaging. A plurality of image sensorsfor monochrome imaging may be used. Note that, as the imaging element,for example, an imaging element that can support capturing of ahigh-resolution image of 4K or more may be used. By obtaining the imageof the operation site with high resolution, the operator 667 can graspthe state of the operation site in more detail, and can progress thesurgery more smoothly.

Furthermore, the imaging element constituting the imaging unit 609 maybe configured to include a pair of imaging elements for acquiring imagesignals for the right eye and the left eye, the image signals beingadapted for 3D display. By performing 3D display, the operator 667 canmore accurately grasp the depth of the living tissue in the operationsite. Note that, in a case where the imaging unit 609 is configured as adual plate type, a plurality of the lens units 607 are providedcorresponding to the respective imaging elements.

Moreover, the imaging unit 609 is not necessarily provided in the camerahead 605. For example, the imaging unit 609 may be provided immediatelyafter the objective lens inside the lens barrel 603.

The drive unit 611 includes an actuator, and moves the zoom lens and thefocus lens in the lens unit 607 by a predetermined distance along theoptical axis under the control of the camera head control unit 615. As aresult, the magnification and focus of the image captured by the imagingunit 609 can be appropriately adjusted.

The communication unit 613 includes a communication device fortransmitting and receiving various types of information to and from theCCU 639. The communication unit 613 transmits the image signal obtainedfrom the imaging unit 609 as RAW data to the CCU 639 via thetransmission cable 665. At this time, in order to display the capturedimage of the operation site with as little delay as possible, the imagesignal is preferably transmitted by optical communication. This isbecause, at the time of surgery, the operator 667 performs surgery whileobserving the state of an affected part with the captured image, andthus, for safer and more reliable surgery, it is required to display amoving image of the operation site in real time as much as possible. Ina case where optical communication is performed, the communication unit613 includes a photoelectric conversion module that converts an electricsignal into an optical signal. The image signal is converted into anoptical signal by the photoelectric conversion module and thentransmitted to the CCU 639 via the transmission cable 665.

Furthermore, the communication unit 613 receives a control signal forcontrolling the drive of the camera head 605 from the CCU 639. Thecontrol signal includes, for example, information related to imagingconditions such as information for specifying a frame rate of a capturedimage, information for specifying imaging conditions (a shutter speed,an aperture, a gain, and the like) at the time of imaging, and/orinformation for specifying a magnification and a focus of a capturedimage. The communication unit 613 provides the received control signalto the camera head control unit 615. Note that the control signal fromthe CCU 639 may also be transmitted by optical communication. In thiscase, the communication unit 613 includes a photoelectric conversionmodule that converts an optical signal into an electric signal, and thecontrol signal is converted into an electric signal by the photoelectricconversion module and then provided to the camera head control unit 615.

Note that the imaging conditions such as the frame rate, the exposurevalue, the magnification, and the focus described above areautomatically set by the control unit 663 of the CCU 639 on the basis ofthe acquired image signal. That is, a so-called auto exposure (AE)function, an auto focus (AF) function, and an auto white balance (AWB)function are implemented by the CCU 639 and the endoscope 601.

The camera head control unit 615 controls the drive of the camera head605 on the basis of the control signal from the CCU 639 received via thecommunication unit 613. For example, the camera head control unit 615controls the drive of the imaging element in the imaging unit 609 on thebasis of information to specify a frame rate of a captured image and/orinformation to specify a shutter speed or an aperture at the time ofimaging. Furthermore, for example, the camera head control unit 615appropriately moves the zoom lens and the focus lens of the lens unit607 via the drive unit 611 on the basis of the information to specifythe magnification and focal point of the captured image. The camera headcontrol unit 615 may further have a function of storing information foridentifying the lens barrel 603 and the camera head 605.

Note that by arranging the lens unit 607, the imaging unit 609, and thelike in a sealed structure with high airtightness and waterproofness,the camera head 605 can have resistance to autoclave sterilizationprocessing.

Next, the functional configuration of the CCU 639 will be described. Thecommunication unit 659 includes a communication device for transmittingand receiving various types of information to and from the camera head605. The communication unit 659 receives an image signal transmittedfrom the camera head 605 via the transmission cable 665. At this time,as described above, the image signal can be suitably transmitted byoptical communication. In this case, the communication unit 659 includesa photoelectric conversion module that converts an optical signal intoan electric signal. The communication unit 659 provides the image signalconverted into the electric signal to the image processing unit 661.

Further, the communication unit 659 transmits a control signal forcontrolling the drive of the camera head 605 to the camera head 605. Thecontrol signal may also be transmitted by optical communication.

The image processing unit 661 performs various types of image processingon the image signal that is RAW data transmitted from the camera head605. Examples of the image processing include various types of knownsignal processing such as development processing, high image qualityprocessing (band emphasis processing, super-resolution processing, noisereduction (NR) processing, and/or camera shake correction processing),and/or enlargement processing (electronic zoom processing). In addition,the image processing unit 661 performs detection processing on the imagesignal for performing AE, AF, and AWB.

The image processing unit 661 includes a processor such as a CPU or aGPU, and the processor operates according to a predetermined program, sothat the image processing and the detection processing described abovecan be performed. Note that, in a case where the image processing unit661 includes a plurality of GPUs, the image processing unit 661appropriately divides information related to an image signal, andperforms image processing in parallel by the plurality of GPUs.

The control unit 663 executes various types of control related toimaging of an operation site by the endoscope 601 and display of thecaptured image. For example, the control unit 663 generates a controlsignal for controlling the drive of the camera head 605. At this time,in a case where the imaging conditions are input by a user, the controlunit 663 generates a control signal on the basis of the input by theuser. Alternatively, in a case where the AE function, the AF function,and the AWB function are mounted on the endoscope 601, the control unit663 appropriately calculates optimum exposure conditions, a focallength, and a white balance on the basis of a result of the detectionprocessing by the image processing unit 661, and generates a controlsignal.

Moreover, the control unit 663 causes the display device 641 to displaythe image of the operation site on the basis of the image signalsubjected to the image processing by the image processing unit 661. Atthis time, the control unit 663 recognizes various types of objects inthe operation site image using various types of image recognitiontechnologies. For example, the control unit 663 can recognize a surgicaltool such as forceps, a specific living body site, bleeding, mist at thetime of using the energy treatment tool 621, and the like by detectingthe shape of the edge, the color, and the like of the object included inthe operation site image. When displaying the image of the operationsite on the display device 641, the control unit 663 superimposes anddisplays various types of surgery support information on the image ofthe operation site using the recognition result. The surgery supportinformation is superimposed and displayed, and presented to the operator667, so that the surgery can be more safely and reliably advanced.

The transmission cable 665 connecting the camera head 605 and the CCU639 is an electric signal cable compatible with electric signalcommunication, an optical fiber compatible with optical communication,or a composite cable thereof.

Here, wired communication is performed using the transmission cable 665in the illustrated example, but communication between the camera head605 and the CCU 639 may be performed wirelessly. In a case where thecommunication between the devices is performed wirelessly, it is notnecessary to lay the transmission cable 665 in the operating room, sothat a situation in which the movement of the medical staff in theoperating room is hindered by the transmission cable 665 can beeliminated.

An example of the endoscopic surgical system 600 to which the technologyaccording to the present disclosure can be applied has been describedabove. Note that, here, the endoscopic surgical system 600 has beendescribed as an example, but the system to which the technologyaccording to the present disclosure can be applied is not limited tosuch an example. For example, the technology according to the presentdisclosure may be applied to a flexible endoscope system for examinationor a microscopic surgery system.

Note that the present disclosure is not limited to the above, and thetechnology described above according to the present disclosure can beapplied within a range not departing from the basic idea of the medicalobservation system according to one embodiment of the presentdisclosure. As a specific example, the technology described aboveaccording to the present disclosure can be appropriately applied notonly to a system to which the endoscope or the operation microscopedescribed above is applied, but also to a system in which an image of anaffected part is captured by an imaging device of a desired form toenable observation of the affected part.

In addition, it is needless to mention that a method of observing anaffected part and a procedure to be applied are not particularlylimited. For example, as an observation method (a treatment method) inwhich an aneurysm is an affected part to be observed, a method using astent or a method using a flow diverter is known in addition to theclipping method described above. In addition, the treatment tool to beused may be different depending on the observation method and theprocedure to be applied. Even in such a case, for example, in the caseof the treatment tool held near the affected part, the technologyaccording to the present disclosure is applied and the motion of thetreatment tool is extracted from sequentially captured images of theaffected part, so that the motion of the affected part can be detected.

As the second application example, an example in a case where themedical observation system according to one embodiment of the presentdisclosure is configured as the endoscopic surgical system including theendoscope unit has been described with reference to FIGS. 11 and 12.

5. CONCLUSION

As described above, in the medical image processing apparatus accordingto one embodiment of the present disclosure, the image processing unitacquires a first fluorescence image captured between the timing when thefirst drug is administered and the timing when the second drug isadministered and a second fluorescence image captured after the timingwhen the second drug is administered. In addition, the image processingunit generates an output image in which fluorescence generated by thesecond drug is enhanced on the basis of the first fluorescence image andthe second fluorescence image.

With the above configuration, under a situation in which theadministration of a fluorescent agent and fluorescence imaging arerepeatedly performed, it is possible to ensure visibility equivalent tothat at the time of observation after the first administration of thefluorescent agent, even at the time of observation after the second andsubsequent administrations of the fluorescent agent. That is, accordingto the medical observation system of one embodiment of the presentdisclosure, it is possible to observe an image acquired in response to adrug in a more suitable manner even in a situation in which the drug isadministered a plurality of times.

Note that, a case where an image is generated on the basis of an imagingresult of the imaging unit has been mainly described above, but theapplication target of the technology according to the present disclosureis not necessarily limited. As a specific example, the technologyaccording to the present disclosure can also be applied to a system thatperforms signal processing on an optical signal such as fluorescenceemitted from a fluorescent material or visible light. In this case, apart that acquires an optical signal corresponding to a detection resultof light arriving from an observation target (for example, an affectedpart) in response to the administration of a drug (for example, a sensorthat detects the light or a part that acquires a detection result fromthe sensor) corresponds to an example of “acquisition unit”. Inaddition, the configuration corresponding to the signal processing unit(CCU) in the medical observation system according to the embodiment andthe modification described above extracts the optical signal generatedby a second drug on the basis of a first optical signal acquired betweenthe timing when the first drug is administered and the timing when thesecond drug is administered and a second signal acquired after thetiming when the second drug is administered. Note that the extraction ofthe optical signal generated by the second drug can be achieved on thebasis of an idea similar to that of the method related to the generationof a fluorescence image in which the fluorescence generated by thesecond drug (for example, the fluorescent agent administered for thesecond and subsequent times) is enhanced in the medical observationsystem according to the embodiment and the modification described above.Furthermore, a configuration corresponding to the signal processing unitin this case corresponds to an example of “optical signal extractionunit”. In addition, the above system corresponds to an example of“medical signal acquisition system”.

Further, the above description has been given mainly focusing onfluorescence imaging performed using a fluorescent agent, but theapplication target of the technology according to the present disclosureis not necessarily limited. That is, the technology according to thepresent disclosure can be applied to an observation method in which adrug is administered to a patient and light in a wavelength bandcorresponding to the drug is observed. As a specific example, it is alsopossible to apply the technology according to the present disclosure toan observation method of administering a contrast medium other than afluorescent agent to a patient and observing an image based on adetection result of light emitted by a material contained in the agent.

Although the preferred embodiments of the present disclosure have beendescribed in detail with reference to the accompanying drawings, thetechnical scope of the present disclosure is not limited to suchexamples. It is obvious that a person having ordinary skill in thetechnical field of the present disclosure can conceive various types ofchanges or modifications within the scope of the technical ideadescribed in the claims, and it is to be understood that these alsobelong to the technical scope of the present disclosure.

Furthermore, the effects described in the present specification are onlyillustrative or exemplary, and are not restrictive. That is, thetechnology according to the present disclosure can exhibit other effectsobvious to those skilled in the art from the description of the presentspecification together with or instead of the above effects.

Note that the following configurations also belong to the technicalscope of the present disclosure.

-   (1)

A medical image processing apparatus comprising

an image processing unit configured to

acquire a first fluorescence image captured between a timing when afirst drug is administered and a first timing when a second drug isadministered and a fluorescence intensity of a predetermined regionstarts to increase, and a second fluorescence image captured after thefirst timing and

generate an output image in which fluorescence generated by the seconddrug is enhanced on a basis of the first fluorescence image and thesecond fluorescence image.

-   (2)

The medical image processing apparatus according to (1), wherein thesecond drug is the first drug.

-   (3)

The medical image processing apparatus according to (1) or (2), whereina timing when each of the first drug and the second drug is administeredis detected on a basis of a fluorescence image captured.

-   (4)

The medical image processing apparatus according to (3), wherein theimage processing unit is configured to detect a minimum value of achange in luminance in time series in at least a part of an image afteran administration of the second drug, and acquire a fluorescence imagecaptured before a timing when the minimum value is detected as the firstfluorescence image.

-   (5)

The medical image processing apparatus according to (1) or (2), whereina timing when each of the first drug and the second drug is administeredis detected on a basis of an input of a user.

-   (6)

The medical image processing apparatus according to any one of (1) to(5) further comprising a storage unit, wherein

the image processing unit is configured to

extract a difference between frames of a captured image according to animaging result of an affected part for each frame by an imaging unit asa difference image and accumulate the difference image in the storageunit,

output an output image corresponding to a result of accumulation of thedifference image in the storage unit, and

output an output image in which fluorescence generated by the seconddrug is enhanced on the basis of the difference image accumulated aftera timing when the second drug is administered by excluding thedifference image accumulated in the storage unit before the timing froman output target as the output image.

-   (7)

The medical image processing apparatus according to (6), wherein theimage processing unit is configured to estimate a relative motion of theimaging unit and the affected part between frames on a basis of animaging result of the affected part for each frame, correct a shiftbetween a first difference image corresponding to an extraction resultof the difference and a second difference image accumulated in thestorage unit on a basis of an estimation result of the motion, andaccumulate the first difference image corrected in the storage unit asthe second difference image that is new.

-   (8)

The medical image processing apparatus according to (7), wherein theimage processing unit is configured to output the output image accordingto an imaging result of the affected part in which light belonging to afirst wavelength band corresponding to at least one of the first drugand the second drug is set as an imaging target, and estimate the motionaccording to an imaging result of the affected part in which lightbelonging to a second wavelength band different from the firstwavelength band is set as an imaging target.

-   (9)

The medical image processing apparatus according to (8), wherein thelight belonging to the second wavelength band includes light belongingto a visible light wavelength band.

-   (10)

The medical image processing apparatus according to any one of (1) to(5), further comprising

a storage unit, wherein

the image processing unit is configured to

hold the first fluorescence image in the storage unit,

extract a difference between the second fluorescence image and the firstfluorescence image, and

output the output image corresponding to an extraction result of thedifference.

-   (11)

The medical image processing apparatus according to any one of (1) (10),wherein the image processing unit is configured to output

a first output image based on the first fluorescence image and

a second output image in which fluorescence generated by the second drugis enhanced on a basis of the first fluorescence image and the secondfluorescence image.

-   (12)

A method of driving a medical image processing apparatus comprisingcausing a computer to

acquire a first fluorescence image captured between a timing when afirst drug is administered and a first timing when a second drug isadministered and t of a predetermined region starts to increase, and asecond fluorescence image captured after the first timing, and

generate an output image in which fluorescence generated by the seconddrug is enhanced on a basis of the first fluorescence image and thesecond fluorescence image.

-   (13)

A medical imaging system comprising:

a light source configured to emit excitation light of a fluorescentmaterial contained in a drug to be administered to a patient;

an imaging unit configured to receive and capture an image of lightincluding fluorescence generated by the drug; and

an image processing apparatus configured to generate an output image inwhich fluorescence generated by a second drug is enhanced on a basis ofa first fluorescence image captured between a timing when a first drugis administered and a first timing when the second drug is administeredand a fluorescence intensity of a predetermined region starts toincrease, and a second fluorescence image captured after the firsttiming.

-   (14)

The medical imaging system according to (13) further comprising

an endoscope unit including a lens barrel to be inserted into a bodycavity of the patient, wherein

the imaging unit captures an image corresponding to a result ofcondensation of light including the fluorescence by the endoscope unit.

-   (15)

The medical imaging system according to (13), further comprising

a microscope unit configured to acquire an enlarged image correspondingto a result of condensation of light including the fluorescence, wherein

the imaging unit captures the enlarged image acquired by the microscopeunit.

-   (16)

A medical image processing apparatus comprising

an image processing unit configured to output an output image thatcorresponds to an imaging result of an affected part of a patient by animaging unit and in which light belonging to a wavelength bandcorresponding to a drug to be administered to the patient is set as animaging target, wherein

the image processing unit includes, as an operation mode,

a first mode for outputting an output image corresponding to an imagingresult of the affected part at that time, and

a second mode for outputting an output image in which fluorescencegenerated by a second drug is enhanced on a basis of a firstfluorescence image captured between a timing when a first drug isadministered and a first timing when the second drug is administered anda fluorescence intensity of a predetermined region starts to increase,and a second fluorescence image captured after the first timing.

-   (17)

A medical signal acquisition system comprising:

a light source configured to emit light in a wavelength bandcorresponding to a drug to be administered to a patient;

an acquisition unit configured to acquire an optical signal belonging toa wavelength band corresponding to the drug; and

an optical signal extraction unit configured to extract an opticalsignal generated by a second drug on a basis of a first optical signalacquired between a timing when a first drug is administered and a firsttiming when the second drug is administered and a fluorescence intensityof a predetermined region starts to increase, and a second opticalsignal acquired after the first timing.

REFERENCE SIGNS LIST

-   11, 12, 13 MEDICAL OBSERVATION SYSTEM-   110 SIGNAL PROCESSING UNIT-   111 DEVELOPMENT PROCESSING UNIT-   112 MOTION ESTIMATION UNIT-   113 DIFFERENCE EXTRACTION UNIT-   114 ADDITION UNIT-   115 FRAME MEMORY-   116, 127, 134 ALIGNMENT PROCESSING UNIT-   120 SIGNAL PROCESSING UNIT-   121 DEVELOPMENT PROCESSING UNIT-   122 DEVELOPMENT PROCESSING UNIT-   123, 133 MOTION ESTIMATION UNIT-   124 DIFFERENCE EXTRACTION UNIT-   125 ADDITION UNIT-   126 FRAME MEMORY-   130 SIGNAL PROCESSING UNIT-   131 DEVELOPMENT PROCESSING UNIT-   132 FRAME MEMORY-   135 DIFFERENCE EXTRACTION UNIT-   191 IMAGING UNIT

1. A medical image processing apparatus comprising an image processingunit configured to acquire a first fluorescence image captured between atiming when a first drug is administered and a first timing when asecond drug is administered and a fluorescence intensity of apredetermined region starts to increase, and a second fluorescence imagecaptured after the first timing and generate an output image in whichfluorescence generated by the second drug is enhanced on a basis of thefirst fluorescence image and the second fluorescence image.
 2. Themedical image processing apparatus according to claim 1, wherein thesecond drug is the first drug.
 3. The medical image processing apparatusaccording to claim 1, wherein a timing when each of the first drug andthe second drug is administered is detected on a basis of a fluorescenceimage captured.
 4. The medical image processing apparatus according toclaim 3, wherein the image processing unit is configured to detect aminimum value of a change in luminance in time series in at least a partof an image after an administration of the second drug, and acquire afluorescence image captured before a timing when the minimum value isdetected as the first fluorescence image.
 5. The medical imageprocessing apparatus according to claim 1, wherein a timing when each ofthe first drug and the second drug is administered is detected on abasis of an input of a user.
 6. The medical image processing apparatusaccording to claim 1 further comprising a storage unit, wherein theimage processing unit is configured to extract a difference betweenframes of a captured image according to an imaging result of an affectedpart for each frame by an imaging unit as a difference image andaccumulate the difference image in the storage unit, output an outputimage corresponding to a result of accumulation of the difference imagein the storage unit, and output an output image in which fluorescencegenerated by the second drug is enhanced on a basis of the differenceimage accumulated after the first timing by excluding the differenceimage accumulated in the storage unit before the first timing from anoutput target as the output image.
 7. The medical image processingapparatus according to claim 6, wherein the image processing unit isconfigured to estimate a relative motion of the imaging unit and theaffected part between frames on a basis of an imaging result of theaffected part for each frame, correct a shift between a first differenceimage corresponding to an extraction result of the difference and asecond difference image accumulated in the storage unit on a basis of anestimation result of the motion, and accumulate the first differenceimage corrected in the storage unit as the second difference image thatis new.
 8. The medical image processing apparatus according to claim 7,wherein the image processing unit is configured to output the outputimage according to an imaging result of the affected part in which lightbelonging to a first wavelength band corresponding to at least one ofthe first drug and the second drug is set as an imaging target, andestimate the motion according to an imaging result of the affected partin which light belonging to a second wavelength band different from thefirst wavelength band is set as an imaging target.
 9. The medical imageprocessing apparatus according to claim 8, wherein the light belongingto the second wavelength band includes light belonging to a visiblelight wavelength band.
 10. The medical image processing apparatusaccording to claim 1, further comprising a storage unit, wherein theimage processing unit is configured to hold the first fluorescence imagein the storage unit, extract a difference between the secondfluorescence image and the first fluorescence image, and output theoutput image corresponding to an extraction result of the difference.11. The medical image processing apparatus according to claim 1, whereinthe image processing unit is configured to output a first output imagebased on the first fluorescence image and a second output image in whichfluorescence generated by the second drug is enhanced on a basis of thefirst fluorescence image and the second fluorescence image.
 12. A methodof driving a medical image processing apparatus comprising causing acomputer to acquire a first fluorescence image captured between a timingwhen a first drug is administered and a first timing when a second drugis administered and t of a predetermined region starts to increase, anda second fluorescence image captured after the first timing, andgenerate an output image in which fluorescence generated by the seconddrug is enhanced on a basis of the first fluorescence image and thesecond fluorescence image.
 13. A medical imaging system comprising: alight source configured to emit excitation light of a fluorescentmaterial contained in a drug to be administered to a patient; an imagingunit configured to receive and capture an image of light includingfluorescence generated by the drug; and an image processing apparatusconfigured to generate an output image in which fluorescence generatedby a second drug is enhanced on a basis of a first fluorescence imagecaptured between a timing when a first drug is administered and a firsttiming when the second drug is administered and a fluorescence intensityof a predetermined region starts to increase, and a second fluorescenceimage captured after the first timing.
 14. The medical imaging systemaccording to claim 13 further comprising an endoscope unit including alens barrel to be inserted into a body cavity of the patient, whereinthe imaging unit captures an image corresponding to a result ofcondensation of light including the fluorescence by the endoscope unit.15. The medical imaging system according to claim 13, further comprisinga microscope unit configured to acquire an enlarged image correspondingto a result of condensation of light including the fluorescence, whereinthe imaging unit captures the enlarged image acquired by the microscopeunit.
 16. A medical image processing apparatus comprising an imageprocessing unit configured to output an output image that corresponds toan imaging result of an affected part of a patient by an imaging unitand in which light belonging to a wavelength band corresponding to adrug to be administered to the patient is set as an imaging target,wherein the image processing unit includes, as an operation mode, afirst mode for outputting an output image corresponding to an imagingresult of the affected part at that time, and a second mode foroutputting an output image in which fluorescence generated by a seconddrug is enhanced on a basis of a first fluorescence image capturedbetween a timing when a first drug is administered and a first timingwhen the second drug is administered and a fluorescence intensity of apredetermined region starts to increase, and a second fluorescence imagecaptured after the first timing.
 17. A medical signal acquisition systemcomprising: a light source configured to emit light in a wavelength bandcorresponding to a drug to be administered to a patient; an acquisitionunit configured to acquire an optical signal belonging to a wavelengthband corresponding to the drug; and an optical signal extraction unitconfigured to extract an optical signal generated by a second drug on abasis of a first optical signal acquired between a timing when a firstdrug is administered and a first timing when the second drug isadministered and a fluorescence intensity of a predetermined regionstarts to increase, and a second optical signal acquired after the firsttiming.